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Continued Examination Under 37 C.F.R. § 1.114 

[1] A request for continued examination under 37 C.F.R. § 1.114, including the fee set forth 
in 37 C.F.R. § 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 C.F.R. § 1.1 14, and the fee set forth in 37 C.F.R. § 
1 .17(e) has been timely paid, the finality of the previous Office action has been withdrawn 
pursuant to 37 C.F.R. § 1.1 14. Applicant's submission filed on Dec. 4, 2008 has been entered. 

Amendments & Claim Status 

[2] This office action is responsive to the Amendment Under 37 C.F.R. § 1.116 received on 
Dec. 4, 2008. Claims 1-8 and 10 remain pending. 

Response to Arguments 

[1] Remarks filed Nov. 7, 2008 with respect to claims 1-8 and 10 have been respectfully and 
fully considered, but are not found persuasive. 

Summary of Remarks regarding Claims 1,10 Rejections under 35. U.S.C. § 103 

Said amendment recites that the play event information obtaining section is configured to obtain 
and recognize a characteristic movement of each player. 
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Applicant respectfully submits that Pingali does not disclose or fairly suggest this feature. In 
particular, Pingali only seems to deal with tracking the ball flight, not movement of the respective 
players. As such, Applicant respectfully submits that Pingali does not disclose the claimed 
invention, particularly, recognizing characteristic movements of the players. Applicant further 
points to the title of the Pingali disclosure: Ball Tracking and Virtual Replays for Innovative 
Tennis Broadcasts. (Emphasis added.) 

The play event information in Pingali merely indicates the path of the tennis ball and the location 
of where the hall landed but not indicates characteristics movements of the players. As such, 
Applicant respectfully asks that the rejection be withdrawn. 

Applicant's Remarks at 8-9, Nov. 7, 2008. 

However, Applicant arguing that "movement of the respective players" are characteristic 
movements to be obtained and recognized under claims 1 and 10 is unpersuasive. Claims 1 and 
10 recite "a play event information obtaining section configured to obtain and recognize play 
event information including characteristic movements of each of the players from picture 
information including in the contents. . .", , and docs not correlate any "movement of the 
respective players" with the characteristic movements in the body of the claim. 

Firstly, Pingali recording and tracking ball flight during a tennis match anticipates the 
claim element in question. Ball flight across a tennis court being tracked ("play event 
information" being obtained and recognized) is indicative that a player hit it, the location of 
where the ball it was hit from, the strength of the hit, the direction of the hit, etc (all 
"characteristic movements of each of the players"). The above examples are also additionally 
indicative of "movement of the respective players". The Examiner suggest further limiting to 
differentiate from the prior art of record. 

Secondly, "a play event information obtaining section configured to obtain and 
recognize. . ." (emphasis added) requires only enough that Pingali must have the hardware 
configuration to do so. Pingali discloses the hardware configuration to enable such a section if 
so desired (i.e., computer, video camera, etc), independent of whether it is actually doing so or 
not. 

Claim Rejections - 35 USC § 112 

[3] The following is a quotation of the second paragraph of 35 U.S.C. § 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 
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Indefinite Limitations 

M.P.E.P. § 2143.02(1) reads, in relevant part: 

If a claim is subject to more than one interpretation, at least one of which would render the claim 
unpatentable over the prior art, the examiner should reject the claim as indefinite under 35 U.S.C. 
1 12, second paragraph (see M.P.E.P. § 706.03(d)) and should reject the claim over the prior art 
based on the interpretation of the claim that renders the prior art applicable. Ex parte Ionescu, 
222 USPQ 537 (Bd. Pat. App. & Inter. 1984) (Claims on appeal were rejected on indefiniteness 
grounds only; the rejection was reversed and the case remanded to the examiner for consideration 
of pertinent prior art.). 

Claims 1-8 and 10 are rejected under 35 U.S.C. § 1 12, second paragraph for containing 
indefinite limitations in using the phrase "configured to" (e.g., "a score information obtaining 
section configured to obtain score information. . ." at claim 1, emphasis added). 

A first interpretation allows an apparatus element having hardware configuration capable 
of performing its intended step (i.e., a score information obtaining section have the hardware 
configuration capable of obtaining score information). 1 A second interpretation allows that it is 
in fact doing so. The Examiner has chosen the first interpretation, and in addition suggests 
positively reciting elements and their purpose without any possible ambiguity (e.g., "a score 
information obtaining section configur e d to obtain obtaining score information. . ."). 

Claim Rejections - 35 USC § 103 
[4] The following is a quotation of 35 U.S.C. § 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Pingali in view of Pizano 

[5] Claims 1-2 and 10 are rejected under 35 U.S.C. § 103(a) as being unpatentable over Ball 
Tracking and Virtual Replays for Innovative Tennis Broadcasts , 1 5 International Conference on 
Pattern Recognition, 2000, Proceedings, Vol. 4, pg 152 - 156 (hereinafter "Pingali") in view of 
U.S. Patent No. 6,101,274 (issued Aug. 8, 2000, hereinafter "Pizano"). 

1 See "configure, n.", Merriam-Webster Dictionary, 1 1 th ed. 2008), < http://www.merriam-webster.com/> (citing "to 
set up for operation especially in a particular way") 
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Regarding Claim 1, while Pingali discloses an image recognition apparatus ("[a] general 
purpose computer" at p. 152, left column) 

for recognizing movements of players matched against each other (the tennis 
match involves at least two players) between domains partitioned with such an obstacle 
as net in a sport match or game (the tennis match must have a net and partitioned 
domains) from contents including a television program being telecasted to show the sport 
match or game ("live television broadcasts" at p. 152, left column), an image material in 
an uncompleted state for broadcasting and contents recorded in such a recording medium 
as a VTR (Introduction, p. 152; fig. 2), [intended use; see M.P.E.P. § 2111.02(11)] 
the image recognition apparatus comprising: 

a score information obtaining section ("[vjirtual [rjeplays and [v]isualtization" section at 
p. 154) configured to [see § 103 rejection above] obtain score information ("includes score- 
based queries" at p. 154, right column; ALSO score information includes recorded paths of the 
tennis ball (e.g., fig. 3) ) indicative of scores ("includes score-based queries" at p. 154, right 
column; ALSO the path of the ball is indicative of scores of the players ) of the respective players 
which vary as the sport match or game proceeds; 

a play event information obtaining section ("[vjirtual [rjeplays and [vjisualtization" 
section at p. 154) configured to [see § 103 rejection above] obtain and recognize play event 
information (play event information would comprising the path of the tennis ball, including 
specifically the exact location where the tennis ball hits the court) including characteristic 
movements (the path of the tennis ball "includes" a characteristic movement of each of the 
players; i.e., a tennis ball path approaching the opposite side of the court indicates that a specific 
player hit it, a characteristic movement) of each of the players from picture information {e.g., fig. 
3) included in the contents, the picture information containing images of the obstacle and the 
players {e.g., fig. 6); and 

an image substance recognizing section ("[vjirtual [rjeplays and [vjisualtization" section 
at p. 154) configured to [see § 103 rejection above] make a comparison ( the comparison of the 
tennis ball position before and after it hits the ground must be made to determine the position 
where tennis ball hits the ground ) between a score information item obtained immediately before 
a point in time {e.g., the path of the tennis ball before hitting the ground at fig. 3) of generation of 
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the play event information (e.g., the exact location where the ball hit the ground at fig. 3) and a 
score information item obtained immediately after the point in time (e.g., the path of the tennis 
ball after hitting the ground at fig. 3) and make reference to a result brought by the play event 
information (play event information would comprising the path of the tennis ball, including 
specifically the exact location where the tennis ball hits the court), thereby recognizing a 
substance (the substance being the exact location on the tennis court) of an image provided by 
the play event information, Pingali does not teach wherein the score information itself is 
displayed on a screen. 

Pizano discloses an apparatus for detecting and interpreting textual captions in digital 
video signals (fig. 6) that teaches wherein the score information itself is displayed on a screen 
("3) SPORTS/TRANSPARENT" in fig. 1; the score in fig. 5). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made for the screen of Pingali to include score information as taught by Pizano "to identify 
the collection of video frames that contain text captions.", Pizano, 5:54-56 and "to create a score 
browser which would enable a person to move directly to specific portions of the video", Pizano, 
1:61-63. 

Regarding Claim 2, while Pingali in view of Pizano disclose the image recognition 
apparatus according to Claim 1, Pingali in view of Pizano do not teach wherein the score 
information obtaining section is configured to [see § 103 rejection above] obtain the score 
information from at least one of the picture information included in the contents, sound 
information including commentary voice of a commentator, and data information transmitted as 
multiplexed on radio waves during broadcasting. 

Pizano discloses an apparatus for detecting and interpreting texual captions in digital 
video signals (fig. 6; fig. 7; fig. 10) that teaches wherein a score information obtaining section 
(section responsible for fig. 6, fig. 7, fig. 10) is configured to [see § 103 rejection above] obtain 
the score information (bottom pictures of the score in fig. 10) from at least one of the picture 
information (top picture in fig. 10) included in the contents (the contents being everything 
displayed at the top picture in fig. 10), sound information including commentary voice of a 
commentator, and data information transmitted as multiplexed on radio waves during 
broadcasting. 



Application/Control Number: 10/522,236 
Art Unit: 2624 



Page 7 



It would have been obvious to one of ordinary skill in the art at the time the invention 
was made for the score information obtaining section of Pingali in view of Pizano to include 
configuring to obtain the score information from at least one of the picture information included 
in the contents, sound information including commentary voice of a commentator, and data 
information transmitted as multiplexed on radio waves during broadcasting as taught by Pizano 
"to identify the collection of video frames that contain text captions.", Pizano, Col. 5, lines 54 - 
56 and "to create a score browser which would enable a person to move directly to specific 
portions of the video", Pizano, 1 :61-63. 

Regarding Claim 10, Claim 1 recites identical features as in Claim 10. Thus, 
references/arguments equivalent to those presented above for Claim 1 are equally applicable to 
Claim 10. 

Pingali in view of Pizano and Sudhir 

[6] Claims 3-8 are rejected under 35 U.S.C. § 103(a) as being unpatentable over Pingali in 
view of Pizano and Automatic Classification of Tennis Video for High-level Content-based 
Retrieval Proceedings of the 1998 International Workshop on Content-Based Access of Image 
and Video Databases (CAIVD '98), 1998, pp 81 - 90 (hereinafter "Sudhir"). 

Regarding Claim 3, while Pingali in view of Pizano disclose image recognition 
apparatus according to Claims 1 or 2, further comprising: 

a domain item extracting section ("[vjirtual [rjeplays and [vjisualtization" section at p. 
154) configured to [see §103 rejection above] extract instrument information on an instrument 
moving between the domains to serve as an object of score count in the sport match or game (an 
ace by Sampras in fig. 3 was within the tennis boundary domain, thus a scoring count) 

Pingali in view of Pizano does not teach a domain item extracting section configured to 
extract from the picture information facility information including information on the obstacle, 
information on the domains and information on boundary lines between the domains and an area 
outside the domains, and player's position information indicative of a player's position; rule 
information storage section configured to store rule information on the sport match or game; and 
basic movement storage section configured to store basic movement information on players' 
characteristic movements generalized in a sport of concern, wherein the play event information 



Application/Control Number: 10/522,236 
Art Unit: 2624 



Page 8 



obtaining section includes a play event information determining section configured to determine 
a play event information item on a play event characteristic of each of the players included in the 
picture information as the play event information based on domain items extracted from the 
picture information, the rule information, and the basic movement information stored in the basic 
movement information storage section. 

Sudhir discloses an automatic classification of tennis video for high-level content-based 
retrieval (Introduction, pg 81) that teaches 

a domain item extracting section (fig. 3; Table 2) configured to [see §103 rejection 
above] extract from the picture information facility information including information on the 
obstacle (dashed line in fig. 7), information on the domains and information on boundary lines 
between the domains and an area outside the domains (fig. 7), and player's position information 
indicative of a player's position ("Player Tracking Module" in fig. 1); 

rule information storage section ("court-line detection module" in fig. 1 for storing rules 
on court- line boundaries) configured to [see §103 rejection above] store rule information on the 
sport match or game; and 

basic movement storage section ("Player Tracking Module" in fig. 1) configured to [see § 
103 rejection above] 'store basic movement information on players' characteristic movements 
generalized in a sport of concern, wherein a play event information obtaining section ("High- 
level Reasoning module" in deciding plays like "high-level events like baseline-rallies, passing- 
shots, serve-and-volleying and net-games" in s. 9, p 89) includes a play event information 
determining section configured to [see §103 rejection above] determine a play event 
information item on a play event characteristic of each of the players included in the picture 
information as the play event information based on domain items (the boxes outlining the players 
on the right side of fig. 6 are the domains) extracted from the picture information, the rule 
information (fig. 7), and the basic movement information stored in the basic movement 
information storage section (fig. 1). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made for the apparatus of Pingali in view of Pizano to include a domain item extracting 
section configured to extract from the picture information facility information including 
information on the obstacle, information on the domains and information on boundary lines 
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between the domains and an area outside the domains, and player's position information 
indicative of a player's position; rule information storage section configured to store rule 
information on the sport match or game; and basic movement storage section configured to store 
basic movement information on players' characteristic movements generalized in a sport of 
concern, wherein the play event information obtaining section includes a play event information 
determining section configured to determine a play event information item on a play event 
characteristic of each of the players included in the picture information as the play event 
information based on domain items extracted from the picture information, the rule information, 
and the basic movement information stored in the basic movement information storage section as 
taught by Sudhir as "there is dire need for algorithms that are able to automatically infer high- 
level content from data.", Sudhir, Introduction, p 81. 

Regarding Claim 4, while Pingali in view of Pizano and Sudhir disclose the apparatus 
according to Claim 3, Pingali in view of Pizano and Sudhir do not teach wherein the player's 
position information is position information indicative of a domain containing each of the 
players and the instrument constantly held and used by the player. 

Sudhir discloses an automatic classification of tennis video for high-level content-based 
retrieval (Introduction, pg 81) that teaches wherein the player's position information is position 
information indicative of a domain containing each of the players and the instrument constantly 
held and used by the player (the player, the tennis racquet, and tennis ball at the point of impact 
are all in the boxes (domain) on the right side of fig. 6). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made for the apparatus of Pingali in view of Pizano and Sudhir to include wherein the 
player's position information is position information indicative of a domain containing each of 
the players and the instrument constantly held and used by the player as taught by Sudhir as 
"there is dire need for algorithms that are able to automatically infer high-level content from 
data.", Sudhir, Introduction, p 81. 

Regarding Claim 5, Pingali discloses wherein the domain item extracting section 
("[v]irtual [rjeplays and [v]isualtization" section at p. 154) is configured to [see §103 rejection 
above] extract the player's position information (the origination of the path of the tennis ball in 
fig. 2 and fig. 4 is the player's position information) from the picture information (fig. 4) based 
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on the facility information (coordinates to track/detect/match instruments, obstacle, and 
boundaries in fig. 4 and fig. 2) extracted by the domain item extracting section. 

Regarding Claim 6, Pingali discloses wherein the domain item extracting section 
("[v]irtual [r]eplays and [v]isualtization" section at p. 154) is configured to [see § 103 rejection 
above] extract the instrument information from the picture information (fig. 2, fig. 4, path of the 
tennis ball) based on the facility information (coordinates to track/detect/match instruments, 
obstacle, and boundaries in fig. 4 and fig. 2) and the player's position information (the 
origination of the path of the tennis ball in fig. 2 and fig. 4 is the player's position information) 
extracted by the domain item extracting section. 

Regarding Claim 7, while Pingali in view of Pizano and Sudhir disclose the apparatus 
according to Claim 3, Pingali in view of Pizano and Sudhir do not teach wherein the play event 
information obtaining section includes a play event index information output section configured 
to output plural play event information items determined by the play event information 
determining section as arranged in a time sequence. 

Sudhir discloses an automatic classification of tennis video for high-level content-based 
retrieval (Introduction, p. 81) that teaches wherein the play event information obtaining section 
includes a play event index information output section (Table 3, p. 88) configured to [see § 103 
rejection above] output plural play event information items ("High-level Annotation" column in 
Table 3) determined by the play event information determining section as arranged in a time 
sequence (the table suggests the arrangement of time sequence). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made for the apparatus of Pingali in view of Pizano and Sudhir to include wherein the play 
event information obtaining section includes a play event index information output section 
configured to output plural play event information items determined by the play event 
information determining section as arranged in a time sequence as taught by Sudhir as "there is 
dire need for algorithms that are able to automatically infer high-level content from data.", 
Sudhir, Introduction, p 8 1 . 

Regarding Claim 8, while Pingali in view of Pizano and Sudhir disclose the apparatus 
according to Claim 3, Pingali in view of Pizano and Sudhir do not teach wherein the play event 
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index information output section is configured to output the play event information items 
together with instrument information items in a time sequence. 

Sudhir discloses an automatic classification of tennis video for high-level content-based 
retrieval (Introduction, p. 81) that teaches wherein the play event index information output 
section (Table 3, p. 88) is configured to [see §103 rejection above] output the play event 
information items ("High-level Annotation" column in Table 3) together with instrument 
information items ("BL" is "Baseline", and thus the table indicates the location information from 
which the instrument was during the serve) in a time sequence (the table suggests the 
arrangement of time sequence). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made for the apparatus of Pingali in view of Pizano and Sudhir to include wherein the play 
event index information output section is configured to output the play event information items 
together with instrument information items in a time sequence as taught by Sudhir as "there is 
dire need for algorithms that are able to automatically infer high-level content from data.", 
Sudhir, Introduction, p. 81. 

Conclusion 

[7] Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to DAVID P. RASHID whose telephone number is (571)270-1578. 
The examiner can normally be reached Monday - Friday 7:30 - 17:00 ET. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on (571) 272-74155. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
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like assistance from a USPTO Customer Service Representative or access to the automated 

information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/David P. Rashid/ 
Examiner, Art Unit 2624 



/Bhavesh M Mehta/ David P Rashid 

Supervisory Patent Examiner, Art Unit 2624 Examiner 

Art Unit 26244 



